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Abstract 

We deal here with the issue of determinism versus randomness in time series. One 
wishes to identify their relative weights in a given time series. Two different tools 
have been advanced in the literature to such effect, namely, i) the "causal" entropy- 
complexity plane [Rosso et al. Phys. Rev. Lett. 99 (2007) 154102] and ii) the esti- 
mation of the decay rate of missing ordinal patterns [Amigo et al. Europhys. Lett. 
79 (2007) 50001, and Carpi et al. Physica A 389 (2010) 2020-2029]. In this work 
we extend the use of these techniques to address the analysis of deterministic finite 
time series contaminated with additive noises of different degree of correlation. The 
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chaotic series studied here was via the logistic map (r = 4) to which we added 
correlated noise (colored noise with Power Spectrum, < A; < 2) of varying 
amplitudes. In such a fashion important insights pertaining to the deterministic 
component of the original time series can be gained. We find that in the entropy- 
complexity plane this goal can be achieved without additional computations. 

PACS: 05.45.Tp; 02.50.-r; 05.40.-a; 05.40.Ca; 



1 Introduction 



The concept of low dimensional deterministic chaos, derived from the modern 
theory of nonlinear dynamical systems, has changed our way of understanding 
and analyzing observational data S{t) (time series), leading to a paradigm- 
change from linear to non-linear approaches. Linear methods interpret ob- 
servational signals from an underlying dynamical system that is regarded as 
being governed by a linear regime under which small perturbations lead to 
small effects. Consequently, all irregular behavior must be attributed to ran- 
dom external inputs [1]. However, chaos theory has shown that random inputs 
are not the only possible source of irregularities in a system's outputs. As a 
matter of fact, nonlinear deterministic autonomous equations of motion rep- 
resenting chaotic systems can give origin to very irregular signals. Of course, a 
system which has both nonlinear characteristics and random inputs will most 
likely produce irregular signals as well [1]. 

Chaotic time series are representative of a set of signals exhibiting complex 
non-periodic traces with continuous, broad band Fourier-spectra and display- 
ing exponential sensitivity to small changes in the initial conditions. Clearly, 
signals emerging from chaotic time series occupy a place intermediate be- 
tween (a) predictable regular or quasi-periodic signals and (b) totally irreg- 
ular stochastic signals (noise) which are completely unpredictable. Chaotic 
time series are irregular in time, barely predictable, and exhibit interesting 
structures in the phase space. 

Chaotic systems display "sensitivity to initial conditions" which manifests 
instability everywhere in the phase space and leads to non-periodic motion 
(chaotic time series). They display long-term unpredictability despite the 
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deterministic character of the temporal trajectory. In a system undergoing 
chaotic motion, two neighboring points in the phase space move away ex- 
ponentially rapidly. Let ^i(t) and X2(t) be two such points, located within 
a ball of radius R at time t. Further, assume that these two points cannot 
be resolved within the ball due to poor instrumental resolution. At some later 
time t' the distance between the points will typically grow to |xi(t') — X2(t')| ~ 
|xi(t)— X2(t)| exp{X\t' —t\) , with A > for a chaotic dynamics and A the biggest 
Lyapunov exponent. When this distance at time t' exceeds R, the points be- 
come experimentally distinguishable. This implies that instability reveals some 
information about the phase space population that was not available at earlier 
times [2]. 

The above considerations allow one to think of chaos as an information source. 
Moreover, the associated rate of generated information can be formulated in 
a precise way in terms of Kolmogorov-Sinai's entropy [SPj . The Kolmogorov- 
Sinai entropy measures the average loss of information rate. Its range of val- 
ues goes from zero for regular dynamics, it is a positive number for chaotic 
system and, infinite for stochastic process. In consequence, if a dynamical sys- 
tem exhibits at least one positive Lyapunov exponent and a finite positive 
Kolmogorov- Sinai entropy one can assert that the system is deterministic- 
chaotic. 

Complex time series are very frequent in nature and also in man-made systems. 
The immediate question that emerges in relation to the underlying dynami- 
cal system that gives origin to these time series reads: Is the system chaotic 
(low- dimensional deterministic) or stochastic? Answering this question is im- 
portant for a proper physical description of irregular dynamics. If one is able 
to show that the system is dominated by low-dimensional deterministic chaos, 
then only few (nonlinear and collective) modes are required to describe the 
pertinent dynamics [5]. If not, then the complex behavior could be modelled 
by a system dominated by a very large number of excited modes which are in 
general better described by stochastic or statistical approaches. 

Even if several methodologies for evaluation of Lyapunov exponents and Kol- 
mogorov-Sinai entropies for time-series' analysis have been proposed (see i.e 
PP) their applicability involves taking into account constraints (stationarity, 
time series length, parameters values election for the methodology, etc.) which 
in general make the ensuing results non-conclusive. Thus, new tools for dis- 
tinguishing chaos (determinism) from noise (stochastic) are needed. 

In the early days of chaotic dynamics it was believed that obtaining finite, 
non-integer values for the fractal dimension was a strong indication of the 
presence of deterministic chaos, in opposition to systems whose dynamics are 
governed by stochastic process, thought to display an infinite value for the 
fractal dimension. However, Osborne and Provenzale [S] were able to present. 
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in a seminal paper, a counter-example of the view that for a stochastic process 
one always detects a non- convergence of the correlation dimension (as the 
estimation of fractal dimension) in computed or measured time series. These 
researchers show that curves (time series) generated by inverting power-low 
spectra and random phases (colored random noises) are random fractal paths. 
Their Housdorf dimension is finite, and thus their correlation dimension is 
finite as well [5]. The main reason is that the Grassberger and Procaccia 
approach for the evaluation of the correlation dimension [6J is independent of 
the points-ordering in the signal and thus not able to test the differentiability 
of the curve under study. Thus, the Grassberger and Procaccia method cannot 
distinguish between fractal attractors and fractal curves if the two have the 
same dimension [5]. 

Among the many different tools advanced so as to distinguishing chaotic from 
stochastic time series we can mention: 

(a) The proposal of Sugihara and May [7] based on nonlinear forecasting. The 
main idea is to compare predicted and actual trajectories. In this way, one 
can make tentative distinctions between dynamical chaos and measure- 
ment errors. For a chaotic time series the accuracy of nonlinear forecast 
diminishes for increasing prediction time-intervals (at a rate which yields 
an estimate of the Lyapunov exponent), whereas for uncorrelated noise, 
the forecasting accuracy does not exhibit such dependency [7]. 

(b) Kaplan and Glass [5|U] introduced a test based on the observation that 
the tangent to the trajectory generated by a deterministic system is a 
function of the position in phase space, and therefore, all the tangents to 
a trajectory in a given phase space region will display similar orientations, 
something that is not observed in stochastic dynamics. 

Note that the two previously mentioned methods have in common the fact 
that one has to choose a certain length-scale e and a particular embedding 
dimension D. 

(c) More recently, Kantz and co-workers p^fTT] proposed to classify the sig- 
nal behavior, without referring to any specific model, as stochastic or 
deterministic on a certain scale of resolution e, according to the depen- 
dence of the (e, r) entropy h{e, r) and finite-size Lyapunov exponent A(e) 
on e. Their methodology for distinguishing between chaos and noise is a 
refinement and generalization of the Grassberger and Procaccia one |6] 
for estimating the correlation dimension, and regarding finite values as 
signatures of deterministic behavior. 

(d) The use of quantifiers based in Information Theory which incorporate in 
their evaluation the "time causality" . We must note, that in this method- 
ology a length scale e is chosen (that means the scale at which one measure 
the time series), and also a particular embedding dimension D, however. 
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its interpretation and main roll is completely different than in the previ- 
ous enumerates methodologies (see for details see below). 

The above mentioned quantifiers based on Information Theory concepts, are 
(1) "entropy", (2) "statistical complexity" and (3) the "entropy-complexity 
plane". Judicious use of this plane could i) lead to interesting insights into 
the characteristics of nonlinear chaotic dynamics, and ii) potentially improve 
our understanding of their associated time series. Moreover, these quantifiers 
can be used to detect determinism in time series [12]. In this vein we mention 
that Rosso et al. [12] found that different Information Theory based measures 
(normalized Shannon entropy and statistical complexity) allow for a better 
distinction between deterministic chaotic and stochastic dynamics when the 
so-called "causal" information is incorporated via Bandt and Pompe's (BP) 
methodology [13]. Indeed, new insight into the characterization of theoretical 
and observational time series, based on the Band and Pompe's methodology, 
reveals the emergence of "forbidden/missing patterns" [T^15|16|IT7] . The BP 
approach has been used also to distinguish deterministic behavior (chaos) from 
randomness in finite time series contaminated with observational white noise 
(uncorrelated noise) [T^15|18] by recourse to the analysis of the decay rate of 
the "missing ordinal patterns" as a function of the time series length. 

Additionally, Zanin [19] and Zunino et al. [20] have recently studied the ap- 
pearance of missing ordinal patterns in financial time series. They found evi- 
dence for the existence of deterministic forces in the medium- and long term 
dynamics. Moreover, they propose that an analysis of the number of missing 
patterns' evolution should be a useful tool in order to quantify the randomness 
of certain time-periods within a financial series. Also, the presence of missing 
ordinal patterns has been recently construed as possible evidence of determin- 
istic dynamics in epileptic states. It is suggested in [21] that a missing patterns' 
quantifier could be regarded as a predictor of epileptic absence-seizures. It is 
of the essence to point out that in such researches only non correlated noise 
(white noise) has been considered, which makes the associated results some- 
what incomplete, since the presence of colored noise has not been taken into 
account. 

It becomes necessary then, and such is the objective of this work, to investigate 
from such viewpoint the robustness of the entropy-complexity causality plane, 
that plays a prominent role in some of the above cited discoveries. We intend 
to do this by analyzing the planes's ability to distinguish between noiseless 
chaotic time series and the ones that are contaminated with additive correlated 
noise (noise with power low spectrum f^^). 

The chaotic series studied here were generated by recourse to a logistic map 
to which noise with varying amplitudes was added. The decay rate of missing 
ordinal patterns was also investigated as a tool to distinguish among these 
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systems. 



The present paper is organized as follows: Section [2] gives a brief description of 
the Information Theory quantifiers used in this work. Section |3] presents the 
Bandt and Pompe methodology used for the evaluation of the quantifiers. The 
methodological framework used in this study is delineated in Section HJ Finally 
Sections [5] and |6] present the discussion of the results and the conclusions, 
respectively. 



2 Information Theory based quantifiers 

2.1 Entropy and Statistical Complexity 

The information content of a system is typically evaluated via a probability 
distribution function (PDF) describing the apportionment of some measurable 
or observable quantity. An information measure can primarily be viewed as a 
quantity that characterizes this given probability distribution P. The Shannon 
entropy is very often used as a the "natural" one [22|- Given any arbitrary 
discrete probability distribution P = {pi : i = 1, - ■ ■ , M}, with M the number 
of freedom- degrees, Shannons logarithmic information measure reads 

M 

m=-J2P^HP^) ■ (1) 
i=l 

It can be regarded as a measure of the uncertainty associated to the physical 
process described by P. From now on we assume that the only restriction on 
the PDF representing the state of our system is YljLi Pj = 1 (micro-canonical 
representation). If S[P] = Smm = we are in position to predict with complete 
certainty which of the possible outcomes i, whose probabilities are given by 
Pi, will actually take place. Our knowledge of the underlying process described 
by the probability distribution is maximal in this instance. In contrast, our 
knowledge is minimal for a uniform distribution and the uncertainty is maxi- 
mal, S[-Pe] = Smax- 

It is widely known that an entropic measure does not quantify the degree 
of structure or patterns present in a process |23]. Moreover, it was recently 
shown that measures of statistical or structural complexity are necessary for 
a better understanding of chaotic time series because they are able capture 
their organizational properties [23]. This specific kind of information is not 
revealed by randomness' measures. The opposite extreme perfect order (like a 
periodic sequence) and maximal randomness (fair coin toss) possess no com- 
plex structure and exhibit zero statistical complexity. States between these 



6 



extremes, a wide range of possible degrees of physical structure exists, that 
should be quantified by the statistical complexity measure. Rosso and cowork- 
ers introduced an effective statistical complexity measure (SCM) that is able 
to detect essential details of the dynamics and differentiate different degrees 
of periodicity and chaos [25j. This specific SCM, abbreviated as the MPR one, 
provides important additional information regarding the peculiarities of the 
underlying probability distribution, not already detected by the entropy. 

The MPR-statistical complexity measure is defined, following the seminal, 
intuitive notion advanced by Lopez- Ruiz et al. [2n], via the product 

Cjs[P] = Qj[P,Pe]-'Hs[P] (2) 
of i) the normalized Shannon entropy 

ns[P] = S[P]/Sma. , (3) 



with S^ax = S[Pe] = InM, {0 < Tis < 1) and P^ = {1/M, ■ ■ ■ , 1/M} (the 
uniform distribution) and ii) the so-called disequilibrium Qj. This quantifier 
is defined in terms of the extensive (in the thermodynamical sense) Jensen- 
Shannon divergence JT[P, Pe] that links two PDFs. We have 

Qj[P,P,] = Qo-J[P,Pe] , (4) 



with 

J[P, P,] = S [(P + Pe)/2] - S[P]/2 - S[Pe]/2 . (5) 

Qo is a normalization constant, equal to the inverse of the maximum possible 
value of i7[P, Pe]. This value is obtained when one of the values of P, say Pm, 
is equal to one and the remaining pi values are equal to zero, i.e., 

go = -2|(^^^^^)ln(M + l) -21n(2M) +lnM| . (6) 



The Jensen- Shannon divergence, that quantifies the difference between two 
(or more) probability distributions, is especially useful to compare the symbol- 
composition of different sequences [27j. The complexity measure constructed 
in this way has the intensive property found in many thermodynamic quan- 
tities [2S]. We stress the fact that the statistical complexity defined above is 
the product of two normalized entropies (the Shannon entropy and Jensen- 
Shannon divergence), but it is a nontrivial function of the entropy because it 
depends on two different probabihties distributions, i.e., the one corresponding 
to the state of the system, P, and the uniform distribution, Pg. 
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2.2 Entropy- Complexity plane 



In statistical mechanics one is often interested in isolated systems character- 
ized by an initial, arbitrary, and discrete probability distribution. Evolution 
towards equilibrium is to be described, as the overriding goal. At equilibrium, 
we can think, without loose of generality, that this state is given by the uniform 
distribution Pe- The temporal evolution of the statistical complexity measure 
(SCM) can be analyzed using a diagram of Cjs versus time t. However, it is 
well known that the second law of thermodynamics states that for isolated 
systems entropy grows monotonically with time {dl-Ls/dt > 0) [2B]- This im- 
plies that Tis can be regarded as an arrow of time, so that an equivalent way 
to study the temporal evolution of the SCM is through the analysis of Cjs 
versus Hs- In this way, the normalized entropy-axis substitutes for the time- 
axis. Furthermore, it has been shown that for a given value of Tis, the range 
of possible statistical complexity values varies between a minimum Cmin and 
a maximum Cmax [21], restricting the possible values of SCM in this plane. 

Therefore, the evaluation of the complexity provides additional insight into 
the details of the systems probability distribution, which is not discriminated 
by randomness measures like the entropy [12123]. It can also help to uncover 
information related to the correlational structure between the components of 
the physical process under study [30j. The entropy-complexity diagram (or 
plane), Tis x Cjs, has been used to study changes in the dynamics of a system 
originated by modifications of some characteristic parameters (see for instance 
Refs. [29]|31II32|[33] and references therein). 




2.3 Estimation of the Probability Distribution Function 

In using quantifiers based on Information Theory (like Tis and Cjs), a proba- 
bility distribution associated to the time series under analysis should be pro- 
vided beforehand. The determination of the most adequate PDF is a funda- 
mental problem because P and the sample space Q are inextricably linked. 
Many methods have been proposed for a proper selection of the probability 
space {fl,P). We can mention: (a) frequency counting [S^, (b) procedures 
based on amplitude statistics [35j, (c) binary symbolic dynamics [36], (d) 
Fourier analysis [37] and, (e) wavelet transform [38], among others. Their ap- 
plicability depends on particular characteristics of the data, such as station- 
arity, time series length, variation of the parameters, level of noise contami- 
nation, etc. In all these cases the dynamics' global aspects can be somehow 
captured, but the different approaches are not equivalent in their ability to 
discern all the relevant physical details. One must also acknowledge the fact 
that the above techniques are introduced in a rather "ad hoc fashion" and 
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they are not directly derived from the dynamical properties themselves of the 
system under study. 

Methods for symbolic analysis of time series that discretize the raw series 
and transform it into a sequence of symbols constitute a powerful tool. They 
efficiently analyze nonlinear data and exhibit low sensitivity to noise |39] . 
However, finding a meaningful symbolic representation of the original series 
is not an easy task [lOll^ . To our best knowledge, the Bandt and Pompe 
approach is the only symbolization technique, among those in popular use, 
that takes into account time-causality in the evaluation of the PDF associated 
to the time series of a given system [13]. The symbolic data is i) created 
by associating a rank to the series-values and ii) defined by reordering the 
embedded data in ascending order. Data are reconstructed with an embedding 
dimension D (see definition and methodological details below). In this way it 
is possible to quantify the diversity of the ordering symbols (patterns) derived 
from a scalar time series, evaluating the so called permutation entropy 1-Lsi and 
permutation statistical complexity Cjs (i.e., the normalized Shannon entropy 
and MPR-statistical complexity evaluated for the Bandt and Pompe's PDF). 



3 The Bandt and Pompe methodology 

3.1 PDF based on ordinal patterns 

To use the Bandt and Pompe [13] methodology for evaluating of the proba- 
bility distribution P associated to the time series (dynamical system) under 
study, one starts by considering partitions of the pertinent Z)-dimensional 
space that will hopefully "reveal" relevant details of the ordinal-structure of 
a given one-dimensional time series S = {xt : t = 1, ■ ■ ■ , A^} with embedding 
dimension D > 1. We are interested in "ordinal patterns" of order D [T3E5] 
generated by 

(s) {xs-(D-l), Xs-(D-2), Xs-u Xg) , (7) 

which assigns to each time s the Z)-dimensional vector of values at times 
s, s — 1, ■ ■ ■ , s — {D — l). Clearly, the greater the value, the more information 
on the past is incorporated into our vectors. By "ordinal pattern" related to 
the time (s) we mean the permutation vr = (ro, ri, ■ ■ ■ , ro-i) of [0, 1, ■ ■ ■ , D — 1] 
defined by 

Xs—ro—l ^ Xs—r£)_2 ^ ■ ■ ■ ^ ri ^ Xg—rQ • (8) 
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In order to get a unique result we set ri < rj_i if Xg-n = Xg-n.i- This is 
justified if the values of Xt have a continuous distribution so that equal values 
are very unusual. Otherwise, it is possible to break these equalities by adding 
small random perturbations. 

Thus, for all the D\ possible permutations vr of order D, their associated 
relative frequencies can be naturally computed by the number of times this 
particular order sequence is found in the time series divided by the total 
number of sequences. The probability distribution P = {p(vr)} is defined by 

^{s\s < M - D + 1; (s), has type tt} 
= M-D + 1 • 



In this expression, the symbol jj stands for "number" . 

The procedure can be better illustrated with a simple example; let us as- 
sume that we start with the time series {1,3,5,4,2,5,...}, and we set the 
embedding dimension D = 4. In this case the state space is divided into 
4! partitions and 24 mutually exclusive permutation symbols are considered. 
The first 4-dimensional vector is (1,3,5,4). According to Eq. ([7]) this vec- 
tor corresponds with {xs~3,Xs-2,Xs-i,Xs). Following Eq. (jH]) we find that 
Xs-3 < Xs-2 < a^s < Xg-i- Then, the ordinal pattern which allows us to fulfill 
Eq. (IH]) will be [3,2,0,1]. The second 4-dimensional vector is (3,5,4,2), and 
[0,3, 1,2] will be its associated permutation, and so on. For the computation 
of the Bandt and Pompe PDF we follow the very fast algorithm described by 
Keller and Sinn |35], in which the different ordinal patterns are generated in 
lexicographic ordering. 

The Bandt and Pompe's methodology is not restricted to time series repre- 
sentative of low dimensional dynamical systems but can be applied to any 
type of time series (regular, chaotic, noisy, or reality based), with a very weak 
stationary assumption (for k = D, the probability for Xt < Xt+k should not 
depend on t [13] )■ It also assumes that enough data are available for a cor- 
rect embedding procedure. Of course, the embedding dimension D plays an 
important role in the evaluation of the appropriate probability distribution 
because D determines the number of accessible states D\. It also conditions 
the minimum acceptable length M ^ D\ of the time series that one needs in 
order to work with reliable statistics. 

The main difference between Information Theory quantifiers evaluated with 
the Bandt and Pompe-PDF and other PDFs (like a histogram-PDF), is that 
they are invariant with respect to strictly monotonous distortions of the data. 
This point is very important for the analysis of observed natural time series 
(like for example, sedimentary data related to rainfall by an unknown nonlinear 
function |1S])- However, due to its invariance properties, quantifiers based on 
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the Bandt and Pompe's approach, when apphed to any other series that is 
strictly monotonic in the given data, will yield the same results. Therefore, in 
general, ordinal data processing tools based on ranked numbers lead to results 
that could be more useful for data analysis than those obtained using tools 
based on metric properties. 



3.2 Forbidden and Missing Ordinal Patterns 

As shown recently by Amigo et al. [H|15fl6|IT7] . in the case of deterministic 
one- dimensional maps, not all the possible ordinal patterns can be effectively 
materialized into orbits, which in a sense makes these patterns "forbidden". 
Indeed, the existence of these forbidden ordinal patterns becomes a persistent 
fact that can be regarded as a "new" dynamical property. Thus, for a fixed 
pattern-length (embedding dimension D) the number of forbidden patterns 
of a time series (unobserved patterns) is independent of the series length A^. 
Remark that this independence does not characterize other properties of the 
series such as proximity and correlation, which die out with time [T5p!7] . For 
example, in the time series generated by the logistic map Xk+i = '^Xki)- — Xk), 
if we consider patterns of length D = 3, the pattern {2,1,0} is forbidden. 
That is, the pattern Xk+2 < Xk+i < Xk never appears [IB]. 

Stochastic process could also present forbidden patterns \£7\ . However, in the 
case of either uncorrelated or correlated stochastic processes (white noise vs. 
noise with power low spectrum /"'^ with k > 0, ordinal Brownian motion, 
fractional Brownian motion, and fractional Gaussian noise) it can be numer- 
ically ascertained that no forbidden patterns emerge. Moreover, analytical 
expressions can be derived [48jJ for for some stochastic process (i.e. fractional 
Brownian motion for PDF's based on ordinal patterns with length 2 < D < 4). 

In the case of time series generated by unconstrained stochastic process (uncor- 
related process) every ordinal pattern has the same probability of appearance 
p3|15|16pT] . That is, if the data set is long enough, all the ordinal patterns 
will eventually appear. In this case, when the number of time-series' obser- 
vations becomes long enough the associated probability distribution function 
should be the uniform distribution, and the number of observed patterns will 
depend only on the length A^ of the time series under study. 

For correlated stochastic processes the probability of observing individual pat- 
tern depends not only on the time series length A^ but also on the correlation 
structure [18] . The existence of a non-observed ordinal pattern does not qual- 
ify it as "forbidden", only as "missing", and it could be due to the finite 
length of the time series. A similar observation also holds for the case of real 
data that always possess a stochastic component due to the omnipresence of 
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dynamical noise |l2|43f44] . Thus, the existence of "missing ordinal patterns" 
could be either related to stochastic processes (correlated or uncorrelated) or 
to deterministic noisy processes, which is the case for observational time series. 

Amigo and co-workers P^|T3] proposed a test that uses missing ordinal pat- 
terns to distinguish determinism (chaos) from pure randomness in finite time 
series contaminated with observational white noise (uncorrelated noise). The 
test is based on two important practical properties: their finiteness and noise 
contamination. These two properties are important because finiteness pro- 
duces missing patterns in a random sequence without constrains, whereas 
noise blurs the difference between deterministic and random time-series. The 
methodology proposed by Amigo et al. [15] consists in a graphic comparison 
between the decay of the missing ordinal patterns (of length D) of the time 
series under analysis as a function of the series length A^, and the decay cor- 
responding to white Gaussian noise. This methodology was recently extended 
by Carpi et al. |18j for the analysis of missing ordinal patterns in stochas- 
tic processes with different degrees of correlation: fractional Brownian motion 
(fBm), fractional Gaussian noise (fGn) and, noises with power spectrum 
(PS) and (fc > 0). Specifically, this paper analyzes the decay rate of missing 
ordinal patterns as a function of pattern-length D (embedding dimension) and 
of time series length A^. Results show that for a fixed pattern length, the de- 
cay of missing ordinal patters in stochastic processes depends not only on the 
series length but also on their correlation structures. In other words, missing 
ordinal patters are more persistent in the time series with higher correlation 
structures. Ref. also has shown that the standard deviation of the esti- 
mated decay rate of missing ordinal patterns (a) decreases with increasing D. 
This is due to the fact that longer patterns contain more temporal information 
and are therefore more effective in capturing the dynamics of time series with 
correlation structures. 



4 Application to the logistic map with additive noise 

The logistic map constitutes a canonic example, often employed to illustrate 
new concepts and/or methods for the analysis of dynamical systems. Thus, 
we will use the logistic map with additive correlated noise in order to exem- 
plify the behavior of the normalized Shannon entropy Hs and the Statistical 
Complexity Cjs, both evaluated using a PDF based on the Bandt-Pompe's 
procedure. We will also investigate the decay rate of "missing ordinal pat- 
terns" in both a) the logistic map with additive noise and b) a pure-noise 
series. 

The logistic map is a polynomial mapping of degree 2, F : x„ — )• Xn+i ^9] . 
described by the ecologically motivated, dissipative system represented by the 
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first-order difference equation 

Xn+l = r ■ X„ ■ (1 - Xn) , (10) 

with < x„ < 1 and < r < 4. 

Let 7]^''^ be a correlated noise with f''^ power spectra generated as describe 
in, for instance, [12]. The steps to be followed are enumerated below. 

(1) Using the Mersenne twister generator [50] through the Matlab© rand 
function we generate pseudo random numbers in the interval (—0.5, 0.5) 
with an (a) almost fiat power spectra (PS), (b) uniform PDF, and (c) 
zero mean value. 

(2) Then, the Fast Fourier Transform (FFT) yj is first obtained and then 
multiplied by f^'^^'^, yielding yf; 

(3) Now, yf is symmetrized so as to obtain a real function. The pertinent 
inverse FFT is obtained, after discarding the small imaginary components 
produced by our numerical approximations. 

(4) The resulting noisy time series is re-scaled to the interval [—1, 1], which 
produces a new time series r^*^^) that exhibits the desired power spectra 
and, by construction, is representative of non-Gaussian noises. 

We considered a time series S = {Sn,n = 1, ■ ■ ■ , N} generated by the discrete 
system: 

5„ = x„ + A-r/f , (11) 

in which, x„ is given by the logistic map and r^^^^ G [—1,1] represents a noise 
with power spectrum /~*^ and amplitude A. 

In generating the logistic map's component of our time-series we fixed r = 4 
and started the iteration procedure with a random initial condition. The first 5- 
10^ iterations were considered part of the transient behavior and were therefore 
discarded. After the transient part died out, N = 10^ data were generated. In 
the case of the stochastic component of our time-series we considered < k < 
2 with fc-values changing by the amount Ak = 0.25. The noise amplitude was 
varied between < A < 1 with increments AA = 0.1. Ten noise time series 
of length = 10^ data (different seeds) were generated for each value of the 
fc-exponent. Figures [T] and |5] display the first return map corresponding to a 
typical signal obtained from Eq. ( ITT]) (we show only the first = 10^ data 
points), for the cases of A; = (white noise) and k = 1 respectively. These 
graphs clearly show how the distorting effect of the additive noise on the 
logistic time series increases as the amplitude A grows. 
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5 Results and discussion 



An important quantity for us, called A4.{N, D), is the average number of miss- 
ing ordinal patterns of length D, that is, patterns which are not observed in 
a time series (TS) with data values. As we mentioned before, for pure cor- 
related stochastic processes the probability of observing an individual pattern 
of length D depends on the time series-length A^ and on the correlation struc- 
ture (as determined by the type of noise /c > ). In fact, as shown for noises 
with an /^'^ Power Spectrum in Ref. [IH], as the value of k > augments - 
which implies that correlations grow - increasing values of A^ are needed if 
one wishes to satisfy the "ideal" condition Ai{N, D) = 0. If the time series is 
chaotic but has an additive stochastic component, then one expects that as 
the time series' length A^ increases, the number of "missing ordinal patterns" 
will decrease and eventually vanish. That this may happen does not only de- 
pend on the length A^ but on the de correlation-structure of the added noise 
as well. 

Here from we will fix the pattern-length a.t D = Q and the time series (TS) 
length at A^ = 10^ data. In Fig. |3]the average number of missing patterns 

< M.[N,D) >k=o (mean value and standard deviation over ten time series 
(see Eq. ( [TT]) ) is displayed as a function of the number of TS-data. Ai was 
evaluated as a function of the TS-length 10^ < A^ < 10^ with AA^ = 20. A = 
corresponds to the case of the pure logistic time series and consequently the 
number of missing patterns is constant < Ai >= 645 (see Fig. |31a). From this 
plot we gather that the mean number of missing patterns rapidly decreases as 
the noise amplitude increases, and could vanish when the time series' length 
is large enough. 

We pass now to consider the quantity < M.{N,D) >kjto (see Eq. [TT]) ] cor- 
responding to the TS-number of missing patterns (mean value and standard 
deviation) as a function of the noise amplitude A. This is the purpose of Fig. 
m Due to the finiteness of the record length A^ we find a non zero number 
of missing patterns. The noise correlation effect is more significant for values 
of A; > 1. This graph illustrates on the dependence of the number of missing 
patterns on i) the noise amplitude A and ii) the noise-correlation k. 

Let us now focus our attention on how the average missing ordinal patterns 

< Ai{N, D) > decays as A^ grows (for a fixed value of the pattern length D) 
by recourse to the exponential law advanced by Amigo et al. P^fTB] and Carpi 
et al. [IB], namely, 

A^(A^,D) = 5 ■ exp{-a ■ AT} , (12) 
where a > is the characteristic decay rate and i? is a constant, a is deter- 
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mined by fitting Eq. ( !T2|) to the numerically generated values of < ^A{N, D) > 
(for each k and A) using the least square method. Figure [5] displays the 
a— values for different fc-values as a function of the noise amplitude A. In 
the same plot we also represent, as horizontal lines, the corresponding values 
of the decay rate a pertaining to an strictly noisy TS. In all cases, for the eval- 
uation of M{N,D) we consider 10^ < < 10^ with AA^ = 20 and D = Q. 
The mean value of the coefficient of the exponential fit (the "goodness" of 
the fit) was always larger than 0.95. 

We see in Fig. [Slthat, if A = (logistic series with no noise), the decay rate 
vanishes. For A ^ strictly forbidden patterns do not exist (a > 0). Missing 
patterns are those that had not appeared before due to the finite TS-length' 
effect. When the amplitude of the added noise increases, the number of these 
missing patterns tends to decrease. However, their observation-probability de- 
pends on the time series' length (A^) and on the noise-correlation (k). 

We can regard the additive noise as a perturbation to the logistic data (see 
Figs. [1] and [2]). As a result of this perturbation the original patterns can 
change and so does the PDF associated to the TS. The number of observed 
patterns becomes now a combination of those originated by, respectively, the 
deterministic and the stochastic series' components (their interaction may play 
a role as well). Note that the added noise can destroy true forbidden patterns, 
although it can create also new forbidden patterns. It is clear from Figs. H] 
and E] that, for a fixed number of data, the number of missing patterns tends 
to decrease and the corresponding decay rate a > 0, displays smaller values 
if both the noise amplitude A and the correlation k increase. However, as 
expected, for any given value of k, the logistic series contaminated with noise 
has a lower decay rate than that of the strictly noisy one. If the logistic map 
is contaminated with non correlated noise {k = 0) then, as Fig. O tells us, 
a < a^''^^^ for all noise amplitudes {0 < A < 1) and deterministic infiuences 
on the TS can be established via the distance from the a-decay curve to the 
strictly stochastic horizontal lines. 

In a real situation, of course, the strength of the contaminating noise is un- 
known. However, were we are able to forecast an estimation or at least make 
an educated guess of the power factor k of the contaminating noise, our results 
would motivate us to conjecture that: 

• Given an observational time series of length and missing patterns decay 
rate a^'^^\ and 

• a pure /c- noise time series of the same length A^, with Q,i>'-^"^^^) ^ 

• if a'^'^^^ < Q;(^-"-o'*e)^ then the observational time series can be associated to 
some deterministic process with an additive noise component. 

Summing up, if one employs for the TS-analysis i) quantifiers based on In- 
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formation Theory and ii) the Bandt and Pompe's PDF, several effects have 
to be considered, namely, those associated to the (a) finite number of data 
N; (b) noise contamination level (represented hj A); (c) noise correlation 
(represented by k); and (d) persistence of forbidden patterns linked to the 
deterministic component of the series. 

The missing patterns' quantifier a can be used for practical proposes in order 
to characterize an observational time series. However, given the nature of the 
above mentioned effects, specific quantifiers like "entropy" T-Ls and "complex- 
ity" Cjs could potentially reveal additional important aspects associated to 
the series-PDF that can not be discerned via the behavior of a. We pass now 
to a discussion centered on the two just mentioned quantifiers. 

For each one of the ten time-series generated for each pair {k,A), the normal- 
ized Shannon entropy Hs and MPR statistical complexity measure Cjs were 
evaluated using i) the Bandt and Pompe PDF and ii) the procedure described 
in previous sections. Fig. [6] displays the pertinent values for series of length 
N = 10^ data and D = 6. If A; ~ we see from these graphs that, for increas- 
ing values of the amplitude A, entropy and complexity values change (starting 
from the value corresponding to the pure logistic series, i.e., A = 0) with a 
tendency to approach the values corresponding to pure noise, that is, (Tis ~ 1 
and Cjs ~ 0). As increases, the dependence on the noise-amplitude A tends 
to become attenuated, and it almost disappears for k ^ 2. This is due to the 
effect of the "coloring" (correlations) of the noise, that grows with increasing 
values of k. 

Using the values displayed in Fig. [6] we generate the associated causality 
entropy-complexity plane for each one of the relevant k values and for different 
noise-amphtudes A. These planes are depicted in Fig. [71 We also display in 
these planes values pertaining to the pure-noise time series (with the same 
record length = 10^ data) in the guise of open symbols. The continuous 
lines represent the values of minimum and maximum statistical complexity 
Cmin and Cmax, respectively, evaluated for D = 6. 

Of course, for A = we obtain the purely deterministic value corresponding 
to the logistic time-series, localized approximately at a medium-hight entropic 
coordinate, near the highest possible complexity. Note that such is the typical 
behavior observed for deterministic systems [12]. For a purely uncorrelated 
stochastic process {k = 0) we have Hs = 1 and Cjs = 0. The correlated 
(colored) stochastic processes {k ^ 0) yield points located at intermediate 
values between the curves Cmin and Cmax-, with decreasing values of entropy 
and increasing values of complexity as k grows [12] (see Fig. [6]) . 

Fig. [7| shows that the main effect of the additive noise {A ^ 0) is to shift 
the point representative of zero noise {A = 0) towards increasing values of 
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entropy 1-is and decreasing values of complexity Cjs- This shift defines a curve 
that is located in the vicinity of the maximum complexity C^ax ~curve. Such 
behavior can be linked to the persistence of forbidden patterns because they, 
in turn, imply that the deterministic logistic component is still operative and 
influencing the TS-behavior. 

We stress the fact that noise acts here as an additive perturbation to the lo- 
gistic series. We may assert that some degree of robustness of our (causality) 
entropy-complexity plane against the effects of TS-noise contamination be- 
comes evident. Moreover, a clear differentiation can be appreciated between 
the localization of the points corresponding to the time series with additive 
noise with those from a "pure-noise" series. We insist on the fact that such 
entropy-complexity plane's results are in agreement with the persistence of 
forbidden patterns from the deterministic component of the time-series. 

Interestingly enough, if we plot these values (for all A) in a single graph, 
as in Fig. |8l we clearly appreciate the behavior described in the preceding 
paragraphs. Moreover, all the pertinent points yield a curve that closely ap- 
proaches the maximum complexity one Cmax- We can hypothesize that this 
curve is characteristic of the dynamical deterministic system, and could be 
used in order to determine the level of noise-contamination. Research in this 
direction is being currently undertaken. 



6 Conclusions 

We have looked carefully at the concept of forbidden/missing ordinal pat- 
terns. Why? Because it has been recently used as a tool for the discrimination 
between deterministic and stochastic behavior in observational time series 
|14|15|16|20|21|19j . Given the importance of the subject, that in the light of 
these previous efforts becomes evident, in the present work we have extended 
the analysis of missing ordinal patterns [TTfTS] and linked it to the causality 
entropy-complexity plane [12]. Our considerations were made with regards to 
deterministic finite time series contaminated (as a perturbation) with additive 
noises of different degree of correlation. 

We were able to show that, by comparing the decay rate of missing ordinal 
patterns both in a noisy time series and in a pure-noise (with some degree 
of correlation) one, insights pertaining to the deterministic component of the 
original time series can be gained. 

This seemingly worthy goal was achieved, without the need of additional nu- 
merical work such as the one needed for evaluating the decay rate of missing 
ordinal patterns. It just suffices to look carefully at the planar locations of 
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the pertinent points, for each time series. Moreover, we have encountered a 
characteristic behavior that can be associated to variations in the amount of 
noise-contamination which, in turn, facihtates the identification of the time- 
series' deterministic component. 
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Fig. 1. First return map corresponding to a typical signal from a noise-contaminated 
logistic map (r = 4). The additive noise has a power spectrum of the f~^-type 
{N = 10^ data). We consider the case k = (white noise) with noise amplitude 
0<A<1. 
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Fig. 2. Same as in Fig. 1 but for A; = 1. 
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Fig. 3. a) Average over ten time series of the missing patterns number 
< M{N,D) > (mean^SD), as a function of the time series length 10^ < iV < 10^ 
for the case A; = (withe noise) and D = 6. Note that for the case A = 
we have a purely logistic time series and the associated number of missing pat- 
terns < Pattern >= 645 is independent of the time series length, b) Details over 
10^ < iV < 10''. 
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Fig. 4. Average values corresponding to the number of missing patterns 
< A4{N,D) > {mean it SD) for times series of = 10^ data and -D = 6, as a 
function of the noise amphtude A and for several k values. 
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Fig. 5. Decay rate a for the missing patterns of the noise-contaminated logistic 
time series, for different values of /c as a function of the additive noise component 
A. Horizontal lines correspond to the pure noise-value with power spectrum. 
The times series' length is 10'^ < N < 10^ data and the pattern length used is 
D = 6. 
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Fig. 6. Average values (mean± SD) corresponding to (a) the normalized Shannon 
entropy Us and (b) the statistical complexity Cjs, for times series of iV = 10^ data 
and D = 6, as a function of the noise amplitude A and for several k values. 
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Fig. 7. The causality entropy-complexity plane associated to Fig. [6] for different 
values of both k and A (times series length N = 10^ data and D = G). The cor- 
responding values for a purely noisy series {N = 10^ data and D = 6) are also 
displayed in the guise of open symbols. The continuous lines represent the values 
of minimum and maximum statistical complexity Cmin and Cmax, evaluated for the 
case of pattern length D = 6. In each graph, the first point at the west has ^ = 
(to the left) and the last at the east A = 1 (to the right). 
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Fig. 8. (a) The causality entropy-complexity plane for all values displayed in (Fig. 
[6]) corresponding to different of both k and A (times series length N = 10^ data 
and D = 6). Values for the case of strictly noisy time series {N = 10^ data and 
D = 6) are also shown as open symbols. The continuous lines represent the values 
of minimum and maximum statistical complexity Cmin and Cmax evaluated for the 
case of pattern length D = 6. (h) Amplification. 
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